Imaging pickup apparatus, image pickup method, image processing apparatus, image processing method, and image processing program

ABSTRACT

An image pickup apparatus includes a color component acquiring unit that acquires picked-up-image color-component signals representing color components of a picked-up image, a face detecting unit that detects a face, a skin-color-region color-component acquiring unit that, when the picked-up image includes a face, acquires skin-color-region color-component signals based on a face detection result, a skin-color-eliminated-component-signal-information creating unit that creates skin-color-eliminated component-signal information obtained by eliminating the skin-color-region color-component signals, obtained by the skin-color-region color-component acquiring unit, from the picked-up-image color-component signals obtained by the color component acquiring unit, a white-balance-gain setting unit that sets a white balance gain by using the skin-color-eliminated component-signal information, and a white balance control unit that performs white balance adjustment of the color components of the picked-up image by using the white balance gain.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2006-286017 filed in the Japanese Patent Office on Oct.20, 2006, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to image pickup apparatuses, image pickupmethods, image processing apparatuses, image processing methods, andimage processing programs. In particular, the present invention relatesto an image pickup apparatus, an image pickup method, an imageprocessing apparatus, an image processing method, and an imageprocessing program that have white balance functions.

2. Description of the Related Art

In recent years, digital image pickup apparatuses, such as digital stillcameras and digital video cameras, have become increasingly used, andthese products have become increasingly sophisticated. Accordingly,manufacturers of image pickup apparatuses want to increase commercialvalues of their products by providing them with more advanced functions.

An auto white balance function is one of such image pickup apparatusfunctions. In the auto white balance function, an original color ofwhite can be reproduced as white by setting the proportions of RGB (red,green, blue) to be uniform in order to automatically adapt to a changein tint of a light source for a subject.

In general, an image pickup apparatus having an auto white balancefunction performs color control in which an achromatic color portion ona subject is extracted and set to be achromatic. In this control, sincethe color of the subject changes depending on light, a range of colorsin the achromatic color portion under natural light is treated as anachromatic color detection range.

However, this control may mistakenly determine that a skin color isachromatic because a face's skin color is very close to the tint of theachromatic color portion under a low-color-temperature light source. Inparticular, this tendency is high when an image has a large skin colorarea. This results in a shift in white balance gain, thus causing theimage to be blue.

Accordingly, a white balance gain setting method is known (see, forexample, Japanese Unexamined Patent Application Publication No.2003-189325). In this method, when a face is detected by detecting aface included in a picked-up image, by moving, to an area other thanthat of the face, a white-balance-evaluation-value acquisition area forsetting a white balance gain, calculation of face portion white balanceis prevented from being performed.

SUMMARY OF THE INVENTION

In the white balance gain setting method, because thewhite-balance-evaluation-value acquisition area is moved from the faceportion before being set and the white balance evaluation value isacquired, when the picked-up image includes a plurality of faces, areasexcluding the faces are irregular, thus making it difficult to acquire awhite balance evaluation value.

In addition, in a case in which the picked-up image includes a pluralityof faces, when an undetected face exists due to false detection, thereis a problem in that auto white balance processing is affected by theskin color of the undetected face.

The present invention has been made in view of the above-describedcircumstances. It is desirable to provide an image pickup apparatus,image pickup method, image processing apparatus, image processingmethod, and image processing program for facilitating white balancecontrol.

According to an embodiment of the present invention, an image pickupapparatus for picking up an image by using a solid-state image sensingdevice is provided. The image pickup apparatus includes a colorcomponent acquiring unit that acquires picked-up-image color-componentsignals representing color components of the picked-up image, a facedetecting unit that detects a face from the picked-up image, askin-color-region color-component acquiring unit that, when thepicked-up image includes a face, acquires skin-color-regioncolor-component signals based on a face detection result, askin-color-eliminated-component-signal-information creating unit thatcreates skin-color-eliminated component-signal information obtained byeliminating the skin-color-region color-component signals, obtained bythe skin-color-region color-component acquiring unit, from thepicked-up-image color-component signals obtained by the color componentacquiring unit, a white-balance-gain setting unit that sets a whitebalance gain by using the skin-color-eliminated component-signalinformation, and a white balance control unit that performs whitebalance adjustment of the color components of the picked-up image byusing the white balance gain.

According to the image pickup apparatus, the color component acquiringunit acquires picked-up-image color-component signals representing colorcomponents of a picked-up image. The face detecting unit detects a facefrom the picked-up image. When the picked-up image includes a face, theskin-color-region color-component acquiring unit acquiresskin-color-region color-component signals based on a face detectionresult. After that, theskin-color-eliminated-component-signal-information creating unit createsskin-color-eliminated component-signal information obtained byeliminating the skin-color-region color-component signals, obtained bythe skin-color-region color-component acquiring unit, from thepicked-up-image color-component signals obtained by the color componentacquiring unit. The white-balance-gain setting unit sets a white balancegain by using the skin-color-eliminated component-signal information.The white balance control unit performs white balance adjustment of thecolor components of the picked-up image by using the white balance gain.

According to an embodiment of the present invention, when a picked-upimage includes a face, skin-color-region color-component signals basedon a face detection result, and skin-color-eliminated component-signalinformation obtained by eliminating skin-color-region color-componentsignals from picked-up-image color-component signals is created. Theskin-color-eliminated component-signal information is used to set awhite balance gain. Thus, even if the picked-up image includes aplurality of faces, from the face detection result, an area to besubject to detection can be designated. Therefore, white balance controlis facilitated.

In particular, in the case of generating color component integrationvalues by respectively integrating the picked-up-image color-componentsignals, generating skin-color-region-color-component integration valuesby respectively integrating color component signals of a detected face,calculating threshold values by using theskin-color-region-color-component integration values and acquiring colorcomponent integration values of the picked-up image that satisfy thethreshold values, and creates the skin-color-eliminated component-signalinformation by calculating differences between the color componentintegration values and the color component integration values thatsatisfy the threshold values, highly accurate white balance control ispossible.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of an image pickupapparatus according to a first embodiment of the present invention;

FIG. 2 is a flowchart showing the operation of the image pickupapparatus according to the first embodiment;

FIG. 3 is an illustration of a picked-up image;

FIG. 4 is an illustration of the result of face detection on thepicked-up image;

FIG. 5 is a flowchart showing the operation of an image pickup apparatusaccording to a second embodiment of the present invention;

FIG. 6 is a flowchart showing the operation of the image pickupapparatus according to the second embodiment;

FIG. 7 is a graph illustrating a method for calculating skin colorthreshold values;

FIG. 8 is an illustration of the result of detection using thresholdvalues; and

FIG. 9 is a schematic graph showing the operations of the image pickupapparatuses according to the first and second embodiments.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention are described below with referenceto the accompanying drawings.

FIG. 1 is a block diagram showing the configuration of an image pickupapparatus 1 according to a first embodiment of the present invention.

The image pickup apparatus 1 includes an optical block 2, a driver 2 a,a driving unit 2 b, an image sensor 3, a timing generating circuit (TG)3 a, an AFE (analog front end) circuit 4, a WB (white balance) amplifier5, a signal processing circuit 6, a compression/conversion circuit 7, asignal detection unit 8, a system controller 9, an operation unit 10, agraphic interface 11, and a display (image monitor) 11 a.

The optical block 2 includes a lens unit for condensing, to the imagesensor 3, incident light from a light source and light (reflected light)from a subject, a driving mechanism for performing focusing and zoomingby moving the lens unit, a shutter mechanism, and an iris mechanism fordetermining, by adjusting a lens stop depending on a subject'silluminance, the amount of light passing through the lens unit, that is,an exposure.

On the basis of a control signal from the system controller 9, thedriver 2 a outputs a driving signal for controlling driving (such asdiaphragm driving) of the mechanisms in the optical block 2.

The driving unit 2 b drives the driving mechanisms of the optical block2 in response to the driving signal from the driver 2 a.

The image sensor 3 is, for example, a solid-state image sensing deviceof a type such as a CCD (charge coupled device) type or CMOS(complementary metal oxide semiconductor) type. The image sensor 3photoelectrically converts an optical image passing through the opticalblock 2, and sends the converted signal to the AFE circuit 4.

Under the control of the system controller 9, the TG 3 a outputs atiming signal for controlling driving of the image sensor 3.

The AFE circuit 4 includes a holding-and-gain-control circuit 41 and anA/D (analog-to-digital) conversion circuit 42. The AFE circuit 4 isformed as, for example, a single IC (integrated circuit). Theholding-and-gain-control circuit 41 performs sampling and holding on theimage signal output from the image sensor 3 by performing CDS(correlated double sampling) so that a good S/N (signal/noise) ratio canbe maintained, and also performs AGC (auto gain control) processing tocontrol a gain. The A/D conversion circuit 42 outputs a digital imagesignal by performing A/D conversion. A circuit for performing the CDSprocessing may be formed on the same substrate as the image sensor 3.

On the basis of an input auto white balance gain (white balance controlsignal), the WB amplifier 5 generates an image signal having an adjustedwhite balance. Specifically, in white balance adjustment, each of RGBgains is adjusted depending on the white balance control signal, andeach of RGB amplitudes of the image signal output from the A/Dconversion circuit 42 is amplified. The control by the WB amplifier 5 ishereinafter referred to as “auto white balance control”.

The signal processing circuit 6 performs digital signal processing, suchas linear matrix processing, gamma processing, and color-differencematrix processing, on an input image signal.

The signal processing circuit 6 generates video signals (luminance andcolor-difference signals) of the subject by performing various types ofimage quality correction on the image signal from the WB amplifier 5 inaccordance with the control signal from the system controller 9.

The signal processing circuit 6 also has a function of detecting aperson's face from an input image. The signal processing circuit 6detects a region including a person's face from the image signal fromthe WB amplifier 5, and supplies the result of detection to the systemcontroller 9.

The compression/conversion circuit 7 performs compression encoding basedon the luminance and color-difference signals. A signal compressed bythe compression/conversion circuit 7 is recorded as an image file on arecording medium (not shown).

The signal detection unit 8 detects RGB components for white balancecontrol from the signal. The signal detection unit 8 forms each mainportion of a color component acquiring unit and skin-color-regioncolor-component acquiring unit in an image pickup apparatus according toan embodiment of the present invention. Details of the signal detectionunit 8 are described later.

The system controller 9 is a microcontroller including, for example, aCPU (central processing unit), a ROM (read-only memory), and a RAM(random access memory). By executing a program stored in the ROM or thelike, the system controller 9 overall controls portions of the imagepickup apparatus 1, such as the optical block 2, the image sensor 3, theAFE circuit 4, the WB amplifier 5, and the signal processing circuit 6,whereby auto focusing, auto exposure, and white balance areautomatically controlled to generate preferable video signals of thesubject, whose image is picked up. The system controller 9 forms a mainportion of each of a skin-color-eliminated-component-signal-informationcreating unit and white-balance-gain setting unit in an image pickupapparatus according to an embodiment of the present invention.

The operation unit 10 includes, for example, various operation keys suchas a shutter release button, levers, and a dial, and outputs a controlsignal based on a user's input operation to the system controller 9.

The graphic interface 11 generates an image signal for displaying animage on the display 11 a from the image signal supplied from the signalprocessing circuit 6 via the system controller 9. By supplying thegenerated image signal, the graphic interface 11 displays the image onthe display 11 a. The display 11 a is formed by, for example, an LCD(liquid crystal display), and displays images such as camera-throughimages being picked up and playback images based on data recorded on therecording medium (not shown).

In the image pickup apparatus 1, image light (reflected light) from thesubject converges in the optical block 2 and is incident on the imagesensor 3. The image light incident on the image sensor 3 isphotoelectrically converted into an electric signal. The obtainedelectric signal is input to the holding-and-gain-control circuit 41, anda signal obtained by sampling and gain control in theholding-and-gain-control circuit 41 is input to the A/D conversioncircuit 42. The signal input to the A/D conversion circuit 42 isconverted into a digital signal and the digital signal is input to theWB amplifier 5 and the signal detection unit 8.

After RGB components for white balance control are detected from thedigital signal, the result of detection is sent to the system controller9 by the signal detection unit 8.

At this time, the signal processing circuit 6 detects a region includinga person's face for the input digital signal input to the systemcontroller 9. When the face is included, the system controller 9 sets anauto white balance gain value so that an original color of white isrepresented by white.

The WB amplifier 5 generates an image signal having a white balanceadjusted on the basis of the set auto white balance gain value, andinputs the generated image signal to the signal processing circuit 6.The signal processing circuit 6 performs digital signal processing, suchas linear matrix processing, on the input image signal. The processedsignal is compressed and converted by the compression/conversion circuit7, and the obtained signal is recorded as an image file on the recordingmedium (not shown).

Next, the operation of the image pickup apparatus 1 according to thefirst embodiment is described below.

FIG. 2 is a flowchart showing the operation of the image pickupapparatus 1 according to the first embodiment. FIG. 3 is an illustrationof a picked-up image.

In step S1, the system controller 9 sets an entire screen area 21 (seeFIG. 3) in a picked-up image as a signal detection frame, and sends asignal detection request to the signal detection unit 8.

In step S11, by performing signal detection on the entire screen area 21by using a preset parameter for achromatic color extraction, the signaldetection unit 8 calculates and sends integration values (represented byΣR, ΣG, and ΣB) of color component signals and integration area(represented by A) in the entire screen area 21, as entire-screen-areasignal-detection results, to the system controller 9.

In step S2, the system controller 9 acquires and stores theentire-screen-area signal-detection results in the RAM. The integrationarea (A) represents the area (i.e., the number of pixels) of a regionthat is actually subject to integration in the picked-up image.

In step S3, on the basis of a detection signal from the signalprocessing circuit 6, the system controller 9 determines whether a facehas been detected in the entire screen area 21.

If a face has not been detected in the entire screen area 21 (No in stepS3), the system controller 9 proceeds to step S7.

Alternatively, if a face has been detected in the entire screen area 21(Yes in step S3), the system controller 9 creates information foreliminating the face's skin color. Specifically, in step S4, a signaldetection frame for auto white balance control is set in each detectedface area 22 (see FIG. 3), and a signal detection request for the facearea 22 is sent to the signal detection unit 8.

In step S12, the signal detection unit 8 performs signal detection onthe face area 22 by using the same parameter used in step S11, andcalculates and sends color component integration values (represented byΣR_(F), ΣG_(F), and ΣB_(F)) and skin color integration area (representedby A_(F)) in the face's skin color region, as face-area signal-detectionresults, to the system controller 9.

In step S5, the system controller 9 acquires the face-areasignal-detection results. In step S6, by calculating differences of thecolor component integration values (ΣR_(F), ΣG_(F), ΣB_(F)) and skincolor integration area (A_(F)) in the face areas 22 from the colorcomponent integration values (ΣR, ΣG, ΣB) and integration area (A)(obtained in step S2) in the entire screen area 21, the systemcontroller 9 calculates an integration value (skin-color-eliminatedcomponent-signal information) and integration area(skin-color-eliminated component-signal information) for auto whitebalance control. The skin color integration area (A_(F)) represents thearea (the number of pixels) of each region that is actually subject tointegration in the face area 22.

The obtained integration values (ΣR_(W), ΣG_(W), ΣB_(W)) and integrationarea (A_(W)) are represented byΣR _(W) =ΣR−ΣR _(F)ΣG _(W) =ΣG−ΣG _(F)ΣB _(W) =ΣB−ΣB _(F)A _(W) =A−A _(F)

In step S7, the system controller 9 calculates a final auto whitebalance gain for auto white balance control by using the obtainedintegration values. Specifically, when the face has not been detected inthe entire screen area 21, the auto white balance gain is calculated byusing the integration values (ΣR, ΣG, ΣB) and the integration area (A).If the face has been detected in the entire screen area 21, such an autowhite balance gain, as RGB components are equal, is calculated by usingthe integration values (ΣR_(W), ΣG_(W), ΣB_(W)) and the integration area(A_(W)).

Although, in the detection in step S11, an entire image is basically setas the signal detection frame, actually, all pixel signals of the imagemay not be detected. For example, only pixels having luminance valuesequal to or greater than a predetermined luminance threshold value maybe detected. Alternatively, detection regions may be set at someintervals. Similarly, in the detection in step S12, all the pixelsignals in the face area 22 may not be detected. In step S12, forexample, a luminance threshold value different from that in step S11 maybe set as a detection parameter specific to face detection, and onlypixels having luminance values equal to or greater than the setluminance threshold value may be detected. As described above, whendifferent detection parameters are set in steps S11 and S12, in step S7,by calculating the auto white balance gain by using an average valueobtained by dividing the integration values (ΣR_(W), ΣG_(W), ΣB_(W)) bythe integration area (A_(W)), calculation accuracy can be enhanced.

In step S8, the system controller 9 outputs the calculated auto whitebalance gain to the WB amplifier 5. The WB amplifier 5 applies the autowhite balance gain to the input image signal, whereby the white balanceis adjusted.

As described above, in the image pickup apparatus 1 according to thefirst embodiment, when a face is detected by the signal processingcircuit 6, signal detection on the skin color of the face area 22 isperformed, and, by calculating differences of the color componentintegration values (ΣR_(F), ΣG_(F), ΣB_(F)) and skin-color integrationarea (A_(F)) in the face area 22 from the color component integrationvalues (ΣR, ΣG, ΣB) and integration area (A) in the entire screen area21, an integration value and integration area for auto white balancecontrol can be calculated. This excludes an effect of the face's skincolor from calculation of the auto white balance gain. Thus, highlyaccurate auto white balance control is possible.

In addition, even if a picked-up image includes a plurality of faces,the image pickup apparatus 1 responds to detection regions for thefaces, in other words, the image pickup apparatus 1 simply performsdetection with a rectangular signal detection frame. Accordingly,regardless of the number of faces, by designating the face area 22,false control (setting) of the auto white balance gain due to a skincolor effect can be prevented.

Although, in the first embodiment, face detection is performed by thesignal processing circuit 6, it is not limited thereto, and, byproviding a dedicated separate face detection circuit, faces in theentire screen area 21 can be detected.

As described above, when the picked-up image includes a plurality offaces, if an undetected face exists due to false detection, there is aproblem in that the auto white balance processing is affected by theskin color of the undetected face.

FIG. 4 is an illustration of the result of face detection in a picked-upimage.

As shown in FIG. 4, due to false detection, only a right face in theentire screen area 21 is detected and is set as a face area 22. In thiscase, an undetected left face area 23 is out of a range for signaldetection for calculating color component integration values (ΣR_(F),ΣG_(F), ΣB_(F)). Thus, an error occurs in auto white balance gain. Animage pickup apparatus, for solving this problem, according to a secondembodiment of the present invention, is described below.

Regarding the image pickup apparatus according to the second embodiment,differences from the first embodiment are mainly described, and similardescriptions are omitted.

The operation of the image pickup apparatus according to the secondembodiment is described below.

FIGS. 5 and 6 are flowcharts showing the operation of the image pickupapparatus according to the second embodiment.

Steps S1 a to S4 a of the system controller 9 and steps S21 a and S22 aof the signal detection unit 8 in FIG. 5 are respectively similar tosteps S1 to S4 of the system controller 9 and steps S11 and S12 of thesignal detection unit 8 in FIG. 2. Accordingly, their description isomitted below.

After executing step S4 a, in step S5 a, the system controller 9acquires the face-area signal-detection results. In step S6 a, thesystem controller 9 calculates skin color threshold values obtained whencalculated integration values of the face area 22 are plotted on acolor-difference graph. This processing is described by using a (B−Y,R−Y) plane as an example.

FIG. 7 is a graph illustrating a method for calculating skin colorthreshold values.

As shown in FIG. 7, color component integration values, obtained in stepS5 a, in the face detection region, are plotted in a region on thecolor-difference plane which generally represents a skin color. In thiscase, Y-axial maximum value (R−Y_(max)) and minimum value (R−Y_(min)),and X-axial maximum value (B−Y_(max)) and minimum value (B−Y_(min)),placed at a predetermined distance from the plotted coordinates so as tosurround the coordinates, are set as skin color threshold values.

By storing beforehand a plurality of combinations of Y_(max) and Y_(min)in the ROM or the like, and, correspondingly to the plotted coordinates,reading one combination of Y_(max) and Y_(min) (i.e., values whosecoordinates exist in a region defined by the threshold values), the skincolor threshold values can be determined. In addition, the storedcombinations of Y_(max) and Y_(min) may correspond to types of lightsources (e.g., sunlight, an electric bulb, a fluorescent light, etc.)determined beforehand for auto white balance control.

In step S7 a, the system controller 9 sets the entire screen area 21 asa signal detection frame, and sends, to the signal detection unit 8, asignal detection request for the entire screen area 21 for detectionusing the skin color threshold values.

In step S23 a, the signal detection unit 8 uses the skin color thresholdvalues to perform signal detection on the entire screen area 21. Thesignal detection unit 8 calculates and sends, to the system controller9, as entire-screen-area signal-detection results based on the thresholdvalues, color component integration values (represented by ΣR′, ΣG′, andΣB′) and integration area (represented by A′) that satisfy (in otherwords, correspond to values in the region surrounded by the thresholdvalues) threshold values in the entire screen area 21.

FIG. 8 is an illustration of the result of detection using the thresholdvalues.

In processing in step S22 a, in addition to color component integrationvalues of a right face 24, color component integration values of a leftface 25 undetected from a face detection signal in step S3 a areincluded in the entire-screen-area signal-detection results based on thethreshold values.

In step S8 a, the system controller 9 acquires the entire-screen-areasignal-detection results based on the threshold values. In step S9 a,from the color component integration values (ΣR, ΣG, ΣB) and integrationarea (A) (obtained in step S2 a) in the entire screen area 21, bycalculating differences of the color component integration values (ΣR′,ΣG′, ΣB′) and integration value (A′) that satisfy the threshold values,the system controller 9 calculates an integration value(skin-color-eliminated component-signal information) and integrationarea (skin-color-eliminated component-signal information) for auto whitebalance control.

The obtained integration values (represented by ΣR_(W1), ΣG_(W1), andΣB_(W1)) and integration area (represented by A_(W1)) are represented byΣR _(W1) =ΣR−ΣR′ΣG _(W1) =ΣG−ΣG′ΣB _(W1) =ΣB−ΣB′A _(W1) =A−A′

In step S10 a, the system controller 9 calculates a final auto whitebalance gain for auto white balance control by using the obtainedintegration values. Specifically, when the face has not been detected inthe entire screen area 21, the auto white balance gain is calculated byusing the integration values (ΣR, ΣG, ΣB) and the integration area (A).If the face has been detected in the entire screen area 21, the autowhite balance gain is calculated by using the integration values(ΣR_(W1), ΣG_(W1), ΣB_(W1)) and the integration area (A_(W1)).

In step S11 a, the system controller 9 outputs the calculated auto whitebalance gain to the WB amplifier 5.

After that, the WB amplifier 5 performs auto white balance control byusing the auto white balance gain so that R=G=B.

FIG. 9 is a schematic graph showing the operations of the image pickupapparatuses according to the first and second embodiments.

In FIG. 9, reference numeral P1 denotes an integration value, for autowhite balance control, obtained by the operation of the image pickupapparatus according to the second embodiment when an undetected faceexists. Reference numeral P2 denotes an integration value, for autowhite balance control, obtained by the operation of the image pickupapparatus 1 according to the first embodiment when an undetected faceexists. As is found in FIG. 9, in auto white balance control usingintegration value P2, the image is blue because the gain for the Bcomponent is higher than that in auto white balance control usingintegration value P1.

According to the image pickup apparatus according to the secondembodiment, advantages similar to those in the first embodiment areobtained.

According to the image pickup apparatus according to the secondembodiment, on the basis of color information of a detected face region,threshold values defining a skin color region in a picked-up image, andthe threshold values are used to detect a skin color in the entirescreen area 21. If, in step S3 a, an undetected face exists, in theoperation of the image pickup apparatus in steps S6 a to S8 a, a skincolor of the undetected face is extracted, thus realizing auto whitebalance control that is not affected by the skin color.

In addition, according to the image pickup apparatus according to thesecond embodiment, not only colors faces, but also all skin colorssatisfying skin-color threshold values, that is, skin colors (e.g., askin color of a person's arm, a cardboard color, etc.) that cause falsedetection in auto white balance control, can be extracted. Thus, highlyaccurate auto white balance control is realized.

Although, in the second embodiment, threshold values are set so that theskin color region is rectangular, the region is not limited to therectangle, and the threshold values may be set so that the skin regionis elliptic.

As described above, an image pickup apparatus, image pickup method,image processing apparatus, image processing method, and imageprocessing program according to embodiments of the present inventionhave been described with reference to the accompanying drawings, but thepresent invention are not limited to the foregoing embodiments. Theconfiguration of each component can be replaced by an arbitrarycomponent having a similar function. In addition, in an embodiment ofthe present invention, an arbitrary different element and step may beadded.

In addition, an embodiment of the present invention may have two or morearbitrary features in each of the foregoing embodiments.

Image pickup apparatuses to which embodiments of the present inventionare applied are not particularly limited. Such image pickup apparatusesinclude, for example, digital still camera, digital video cameras,cellular phones, and PDAs (personal digital assistants).

Although, in each foregoing embodiment, the present invention is appliedto an image pickup apparatus, it is not limited thereto and may beapplied to image processing apparatuses for processing input images. Theimage processing apparatuses are not particularly limited. For example,they include printers.

The above-described processing functions may be realized by a computer(specifically, by allowing a computer to execute a predetermined imageprocessing program). In this case, a program describing processingdetails of an image pickup apparatus and image processing apparatus isprovided. By executing the program on the computer, the processingfunctions are realized on the computer. The program describing theprocessing details may be recorded on computer-readable recording media.The computer-readable recording media include, for example, magneticrecording devices, optical discs, magneto-optical recording media, andsemiconductor memories. The magnetic recording devices include, forexample, HDDs (hard disk drives), FDs (flexible disks), and magnetictapes. The optical discs include, for example, DVDs (digital versatilediscs), DVD-RAMs (random access memories), CD-ROMs (compact-discread-only memories) and CD-Rs (Recordable), and CD-RWs (ReWritable). Themagneto-optical recording media include, for example, MOs(magneto-optical disc).

When the program is distributed, for example, transportable recordingmedia, such as DVDs and CD-ROMs, containing the program, are sold. Inaddition, by storing the program in a storage device of a servercomputer, the program can be transferred from the server computer toanother computer.

A computer that executes the program stores, in its storage device, forexample, the program contained in a transportable recording medium orthe program transferred from the server computer. The computer reads theprogram from its storage device and executes processing in accordancewith the program. The computer can also execute the processing inaccordance with the program by directly reading the program from thetransportable recording medium. In addition, whenever the program istransferred from the server computer, the computer can sequentiallyexecute processing in accordance with the received program.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image pickup apparatus for picking up an image by using asolid-state image sensing device, the image pickup apparatus comprising:a color component acquiring unit that acquires picked-up-imagecolor-component signals representing color components of the picked-upimage; a face detecting unit that detects a face from the picked-upimage; a skin-color-region color-component acquiring unit that, when thepicked-up image includes a face, acquires skin-color-regioncolor-component signals based on a face detection result; askin-color-eliminated-component-signal-information creating unit thatcreates skin-color-eliminated component-signal information obtained byeliminating the skin-color-region color-component signals, obtained bythe skin-color-region color-component acquiring unit, from thepicked-up-image color-component signals obtained by the color componentacquiring unit; a white-balance-gain setting unit that sets a whitebalance gain by using the skin-color-eliminated component-signalinformation; and a white balance control unit that performs whitebalance adjustment of the color components of the picked-up image byusing the white balance gain.
 2. The image pickup apparatus according toclaim 1, wherein: the color component acquiring unit generates colorcomponent integration values by respectively integrating thepicked-up-image color-component signals; the skin-color-regioncolor-component acquiring unit generatesskin-color-region-color-component integration values by respectivelyintegrating color component signals of the face detected by the facedetecting unit; and theskin-color-eliminated-component-signal-information creating unit createsthe skin-color-eliminated component-signal information by calculatingdifferences between the color component integration values and theskin-color-region-color-component integration values.
 3. The imagepickup apparatus according to claim 1, wherein the skin-color-regioncolor-component acquiring unit calculates color component thresholdvalues defining a skin color region extracted from the face detectionresult, and uses the color component threshold values to acquire colorcomponent signals representing a skin color region in the picked-upimage.
 4. The image pickup apparatus according to claim 3, wherein: thecolor component acquiring unit generates color component integrationvalues by respectively integrating the picked-up-image color-componentsignals; after the skin-color-region color-component acquiring unitgenerates skin-color-region-color-component integration values byrespectively integrating color component signals of the face detected bythe face detecting unit, the skin-color-region color-component acquiringunit calculates the color component threshold values by using theskin-color-region-color-component integration values, and acquires colorcomponent integration values of the picked-up image which satisfy thecolor component threshold values; and theskin-color-eliminated-component-signal-information creating unit createsthe skin-color-eliminated component-signal information by calculatingdifferences between the color component integration values and the colorcomponent integration values of the picked-up image which satisfy thecolor component threshold values.
 5. An image pickup method for pickingup an image by using a solid-state image sensing device, the imagepickup method comprising the steps of: acquiring picked-up-imagecolor-component signals representing color components of the picked-upimage; detecting a face from the picked-up image; when the picked-upimage includes a face, acquiring skin-color-region color-componentsignals based on a face detection result; creating skin-color-eliminatedcomponent-signal information obtained by eliminating theskin-color-region color-component signals from the picked-up-imagecolor-component signals; setting a white balance gain by using theskin-color-eliminated component-signal information; and performing whitebalance adjustment of the color components of the picked-up image byusing the white balance gain.
 6. An image processing apparatus forperforming white balance processing on an input image, the imageprocessing apparatus comprising: a color component acquiring unit thatacquires input-image color-component signals representing colorcomponents of the input image; a face detecting unit that detects a facefrom the input image; a skin-color-region color-component acquiring unitthat, when the input image includes a face, acquires skin-color-regioncolor-component signals based on a face detection result; askin-color-eliminated-component-signal-information creating unit thatcreates skin-color-eliminated component-signal information obtained byeliminating the skin-color-region color-component signals, obtained bythe skin-color-region color-component acquiring unit, from theinput-image color-component signals obtained by the color componentacquiring unit; a white-balance-gain setting unit that sets a whitebalance gain by using the skin-color-eliminated component-signalinformation; and a white balance control unit that performs whitebalance adjustment of the color components of the input image by usingthe white balance gain.
 7. An image processing method for performingwhite balance processing on an input image, the image processing methodcomprising the steps of: acquiring input-image color-component signalsrepresenting color components of the input image; detecting a face fromthe input image; when the input image includes a face, acquiringskin-color-region color-component signals based on a face detectionresult; creating skin-color-eliminated component-signal informationobtained by eliminating the skin-color-region color-component signalsfrom the input-image color-component signals; setting a white balancegain by using the skin-color-eliminated component-signal information;and performing white balance adjustment of the color components of theinput image by using the white balance gain.
 8. An image processingprogram for performing white balance processing on an input image, theimage processing program allowing a computer to function as: a colorcomponent acquiring unit that acquires input-image color-componentsignals representing color components of the input image; a facedetecting unit that detects a face from the input image; askin-color-region color-component acquiring unit that, when the inputimage includes a face, acquires skin-color-region color-componentsignals based on a face detection result; askin-color-eliminated-component-signal-information creating unit thatcreates skin-color-eliminated component-signal information obtained byeliminating the skin-color-region color-component signals, obtained bythe skin-color-region color-component acquiring unit, from theinput-image color-component signals obtained by the color componentacquiring unit; a white-balance-gain setting unit that sets a whitebalance gain by using the skin-color-eliminated component-signalinformation; and a white balance control unit that performs whitebalance adjustment of the color components of the input image by usingthe white balance gain.